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VIRTUAL SENSOR FOR DATA & SENSOR FUSION 



Cross-Ref erence to Related Applications 

[000.5] This application claims the benefit of the 
priority date of Provisional Application No. 60/477,305, 
filed June 10, 2003. 

Field of the Invention 
[0001] This invention relates to fusion of 
information from sensors for .the purpose of classifying 
objects or conditions observed by the sensors. 

Background of the Invention 
[0002] The use of sensors to determine the nature 
or classification of objects is old. Such sensors receive 
raw data from an observed direction or space in either a 
passive or active manner, and process the information 
according to some algorithm in order to make a 
determination of the nature of the object or condition. 
For example, a radar system operating as an active sensor 
may transmit radar signals in a desired direction, and then 
processes signals returned from a target to determine 
various characteristics of the received signal in order to 
characterize the target as, say, an F-15 fighter aircraft 
rather than a C-130 transport aircraft. A passive sensor 
might detect the emissions of an active sensor carried by a 
remote platform, determine the type of sensor detected, and 
identify platforms capable of carrying such an active 
sensor. 

[0003] A great deal of work has been done in the 
field of fusion of the outputs of various sensors 
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associated with a battle region, in order to rationalize 
the results of the many sensors observing the region from 
different vantage points under different operating 
conditions . 

5 [0004] Improved or alternative fusion is desired. 

Summary of the Invention 
[0005] A method according to an aspect of the 
invention is for fusing information from plural sources. 

10 The method comprises the step of observing an object with 
at least first and second sensors, each of which (a) 
evaluates evidence or information and (b) based on the 
evidence, assigns a taxonomic classification to its 
observation of the object. The method further comprises 

15 the step of fusing the evidence from the first and second 
sensors to produce compound evidence. A classification is 
assigned based on the compound evidence. In a particular 
embodiment of the invention, the classification based on 
compound evidence is taxonomic or type classification. 

20 

Brief Description of the Drawing 
[0006] FIGURE 1 is a simplified block diagram of 
a prior-art multiple-sensor fusion arrangement; 

FIGURE 2 is a simplified block diagram of a 

2 5 prior- art sensor of FIGURE 1, showing how evidence provides 

the basis for taxonomic classification; 

FIGURE 3 is a simplified block diagram of a 
sensor according to an aspect of the invention, showing 
that the evidence is available externally; and 

3 0 FIGURE 4 illustrates a virtual sensor 

incorporating fused evidence from plural sensors, together 
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with a taxonomic classifier operating on the fused evidence 
to classify the target or object. 

Description of the Invention 
5 [0007] In FIGURE 1, a system 10 includes a first 

sensor 12, a second sensor 14, and a third sensor 16, each 
observing a region designated generally as 18 which 
happens, at the illustrated time, to contain an object 20. 
For def initeness, the region 18 may be considered to be a 

10 region near a battlefield, object 20 may be an aircraft, 
and each of sensors 12, 14, and 16 may be any type of 
active or passive sensors used to detect and classify 
aircraft. The classification produced by sensor 1 is 
reported at an output port 12o, and is coupled by any 

15 means, specifically as a hard-wired path 22, to a 

classification fusion function illustrated as a block 24. 
Similarly, sensors 14 and 16 each generate a classification 
or determination of the type of object at their output 
ports 14o and 16o, and these determinations are coupled to 

2 0 block 24 by any means, which in FIGURE 1 is illustrated as 
a hard-wired path 26. 

[0008] In the prior- art arrangement of FIGURE 1, 
classification fusion arrangement 24 performs processing in 
known manner to rationalize the determinations made by the 

2 5 various sensors. As a simple example, if a large number of 
sensors report that object 20 is an F-16 aircraft, and one 
sensor reports that it is a battleship flying at altitude, 
block 24 may reject the battleship classification and 
produce a rationalized classification at its output port 

30 24o to the effect that the object 20 is an F-16. 

[0009] In FIGURE 2, exemplary sensor 12 is 
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illustrated as including a block 110 representing one or 
more transducers with their associated electronics. This 
might be, for example, the antenna, transmitter, and 
receiver of a radar system in the context of an active 
5 sensor, or a staring array and its amplifiers in the 
context of a heat sensor. 

[0010] The signals representing raw information 
about the object sensed are applied from block 110 of 
FIGURE 2 to a block 112, which represents processing 

10 performed by the sensor on the raw data to generate 

information which allows classification of the type of 
object. This processing might be spectral processing of 
the received signal in the case of a radar system. The 
processing might also include decoding and information 

15 processing in the case of intercepted data or voice 

transmissions, shape estimation in the case of imaging 
sensors, kinematic information such as acceleration, and 
the like. 

[0011] The evidentiary information produced by 
20 block 112 of FIGURE 2 is applied to a taxonomic (type) 

classification block 114, which makes a determination of 
the object type by comparing the evidence with stored 
information relating to the characteristics of the evidence 
for various types of objects. The final classification is 
25 output from port 12o for transmission over path 22. 

[0012] In FIGURE 3, elements corresponding to 
those of FIGURE 2 are designated by like reference 
numerals. In FIGURE 3, a sensor 312 according to an aspect 
of the invention produces its taxonomic classification at 
3 0 an output port 312o, also includes a further output port 

314 at which the evidence used by taxonomic classification 
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block 114 can be accessed. 

[0013] In FIGURE 4, first, second, and third 
sensors 312 lt 312 2/ and 312 3 are similar to sensor 312 of 
FIGURE 3 . Each of the three sensors of FIGURE 4 observes 
5 object 20. In addition to producing three classifications 
at output ports 312 o:L , 312 o2/ and 312 Q 3 , the externally- 
available evidence from output ports 314 1; 314 2 , and 314 3 is 
coupled by paths, illustrated as hardwired paths 422 and 
426 to an evidence fusion block 424. Evidence fusion block 

10 424 produces fused evidence in any one of a variety of 
prior-art manners, to thereby effectively combine the 
evidence of sensors 312i, 312 2/ and 312 3 into the equivalent 
of a single virtual sensor processor 412. The combined 
evidence is presented to a taxonomic classification block 

15 414, which can be in accordance with any prior-art 

arrangement, and more specifically may be similar to block 
24 of FIGURE 1. Block 414 produces a taxonomic 
classification based on the combined evidence. 

[0014] In general, the calculations are performed 

20 by letting p(E|a) be the likelihood that a sensor produces 
evidence E when an object that the sensor observes has a 
characteristic a. A typical sensor will be able to produce 
more kinds of evidence E n , n = 1, 2, 3, . . . when an object 
with characteristic a is observed. Let p(E n |a) be the 

25 likelihood that a sensor will produce evidence E n when an 

object observed by the sensor has a characteristic a. As a. 
specific example, a sensor finds a dominant frequency of 
100 KHz (kilohertz) when observing a J- 100 aircraft engine, 
but may find a frequency of 12 0 KHz during the observation. 

30 The likelihood p(100 KHZ|j-100) of observing 100 KHz may be 
0.95, and the likelihood p(120 KHz|j-100) = 0.05. In 
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general, objects may have different characteristics a k , k = 
1,2,3,.... A sensor may produce the same evidence E n 
when objects with different characteristics are observed. 
Let p(E n |a k ) be the likelihood that a sensor will produce 
5 evidence E n when an observed object has a characteristic a k . 
As an example, a sensor may find a dominant frequency of 
100 KHz when observing a J-100 engine, but may find the 
same frequency when observing a J-150 engine. The 
likelihood p(100 KHz | J-100) of observing 100 KHz may be 

10 0.95, while the likelihood p(100 KHz | J-150) may be 0.40 
(the sums need not add to 1.00) . Let p-j(E n |a k ) be the 
likelihood that a sensor j produces evidence E n when an 
observed object has a characteristic a k . The invention 
applies to any sensor, termed a "characterized" sensor, for 

15 which pj(E n |a k ) is either known or can be estimated for one 
or more E n and one or more a k . 

[0015] The classifications produced by sensors 
312i, 312 2 , and 312 3 may be used for other purposes or not 
used, as desired. 

2 0 [0016] Other embodiments of the invention will be 

apparent to those skilled in the art. For example, while 
hard-wired paths have been illustrated for coupling the 
evidence from each sensor to a combiner, the paths may be, 
or include, wireless portions, or be encoded, or in. general 

2 5 of any appropriate sort. 

[0017] Thus, in a very general aspect of the 
invention, a plurality of sensors observe an object, and 
the raw sensor data is processed to produce evidence 
signals representative of characteristics which may be used 

3 0 to classify the object as to type. The evidence from the 

plurality of sensors is fused to generate fused or combined 
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evidence. Thus, the fused evidence is equivalent to 
signals produced by a virtual sensor. The fused evidence 
is applied to a taxonomic classifier to determine the 
object type. 

5 [0018] More particularly, a method according to 

an aspect of the invention is for fusing information from 
plural sources (312 lf 312 2/ and 312 3 ) . The method comprises 
the step of observing an object with at least first (312i) 
and second (312 2 ) sensors, each of which (a) evaluates 

10 evidence or information and (b) based on the evidence, 

assigns a taxonomic classification to its observation of 
the object. The method further comprises the step of 
fusing the evidence (block 424) from the first and second 
sensors to produce compound evidence. A classification is 

15 assigned (block 4 14) based on the compound evidence. In a 
particular embodiment of the invention, the classification 
based on compound evidence is taxonomic or type 
classification. 
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